What happened to health labour markets during COVID-19? Insights from the analysis of cross-sectional survey data on the perceptions of medical doctors in Brazil

Objectives To examine physicians’ perceptions of changing employment opportunities in Brazil, and gain an insight into labour markets in low/middle-income countries (LMICs) during the pandemic. Study design Descriptive and inferential analysis of a quantitative dataset from a representative cross-sectional survey of physicians of two Brazilian states. Settings São Paulo and Maranhão states in Brazil. Participants Representative sample of 1183 physicians. Outcome measures We estimated prevalence and 95% CIs for physicians’ perceptions of changes in demand and supply of doctors, as well as changes in prices of medical services for facilities of practice in the two states, stratified by public, private and dual-practice physicians. Results Most doctors reported increased job opportunities in the public sector (54.9%, 95% CI 52.0% to 57.7%), particularly in Maranhão state (65.0%, 95% CI 60.9% to 68.9%). For the private sector, increased opportunities were reported only in large private hospitals (46.7%, 95% CI 43.9% to 49.6%) but not in smaller clinics. We recorded perceptions of slight increases in availability of doctors in Maranhão, particularly in the public sector (51.4%, 95% CI 43.2% to 59.5%). Younger doctors recounted increased vacancies in the public sector (64%, 95% CI 58.1% to 68.1%), older doctors only in walk-in clinics in Maranhão (47.5%, 95% CI 39.9% to 55.1%). Those working directly with patients with COVID-19 saw opportunities in public hospitals (65%, 95% CI 62.3% to 68.4%) and in large private ones (55%, 95% CI 51.8% to 59.1%). Conclusions Our findings hint that health labour markets in LMICs may not necessarily shrink during epidemics, and that impacts will depend on the balance of public and private services in national health systems.

-In relation to data, I would like to see a table summarizing the  main descriptive statistics for each variable and disaggregated by  territory. -Lastly, I believe the analysis could enrich by including a test of means difference between territories.

REVIEWER
Bhandari, Neeraj University of Nevada Las Vegas, Healthcare Administration and Policy REVIEW RETURNED 09-Jun-2023

GENERAL COMMENTS
Manuscript review for BMJ Open Comments to authors Manuscript Number: BMJOpen-2023-075458 "What happened to health labour markets during COVID-19? Insights from a survey of medical doctors in Brazil" June 9, 2023 This study provides some intriguing survey data on perceptions of Brazilian doctors regarding demand for their services during the COVID-19 pandemic. The study is well written and uses simple but rudimentary descriptive methods to present its findings. It uses proxies for capturing perception of doctors, which limits its utility in two ways: perceptions can depart from reality and the specific proxies chosen here may contain some measurement error in and of themselves, especially the supply proxy items. That said the study is a good addition to existing body of literature on pandemic impacts. However, I would caution authors against reaching more sweeping interpretation than their data and methods allow. I am not sure the study results permit a clear inference about differential impacts of the pandemic on labor markets in private vs. the public sector, absent more information on the impacts on diverse settings within the public sector (e.g., primary care, urgent care elective care and inpatient settings). I discuss these (and other more minor) issues in more detail below: 1. It is not clear that the text of the question used to proxy physician supply is cleanly worded ("availability of physicians for recruiting"; Table 1 row 5) or directed at the right kind of subjects to elicit accurate information. Physicians are likely to have a good idea of demand for their own work but supply and recruitment difficulties would be better known by hiring administrators and executives responsible for running hospitals and clinics. Authors briefly allude to this issue in limitation but it needs to be emphasized more directly in interpreting their results. The anomalous finding of price stability given the supply-demand mismatch may be explained by an erroneous measurement of supply.
2. The text of the survey question for A&E remuneration differs between Table 1 (row 5, column 2) and the Datafolha instrument provided in the supplementary appendix (Q 34). The latter question does not refer to A&E shift but to specialty wages. Please address the discrepancy.
3. Please label supplementary tables using a different convention e.g., S2 or A2 to distinguish from tables in the main manuscript.
4. The denotation for confidence interval is "CI" in the text and "IC" in the tables. Please make it consistent.
5. Were any formal statistical comparisons (t-test between group means) between responses for the two states made? I see it alluded in methods section but don't see it discussed in results or described in tables.
6. I understand that responses to survey items included options to choose "unknown" when the respondent was unsure as to the main response options. However, I do not see this option listed in the descriptive data tables. Was this because no respondent picked that option? That would be unusual, especially for supply proxies, where the text is somewhat ambiguous. Relatedly, some of the cells in Table 3 do not add to 100%. For example, response options for overall sample regarding the perception of availability of doctors do not add up to 100% (increased 44.8, decreased 27.7, no change 19.2). Also, total number of respondents is listed as 1181 in some tables, whereas the Sao Paulo (632) and Maranhao respondents (551) total up to 1183. Please clarify these anomalies.
7. From the Datafolha instrument, it is evident that respondents were also asked to quantify the intensity of their initial responses to demand and supply proxy items on a 10-point scale (Q31a, 31b, 33a, 33b). Why was this information not utilized to further evaluate and contextualize their initial responses?
8. Authors interpret their findings as providing evidence that pandemic had differential effects on public sector and private sectors labor markets. I am not so sure. It may also be that pandemic had major impacts on specific acute care clinical settings like hospitals (where demand accelerated due to exploding COVID-19 related morbidity/mortality) regardless of whether these were public hospitals under SUS or private hospitals. Moreover, as authors themselves state, many SUS hospitals are run by private non-profit entities that hire contractual workers. Did SUS facilities that primarily provide elective or primary care buck the trend of reduced work opportunities? If so, that would support the theory of differential impacts but I am not sure whether SUS-related response data for demand proxy items was disaggregated by facility setting type 9. Some statements in the discussion are ambiguously worded and inconsistent with results: Demand and supply of doctors in Sao Paolo and Maranhao appear to have experienced opposite pandemic effects…., Page 10, Lines 59-60). Please review and reword.  Table S1). The sampling was explained in the Methods (Study settings and data collection, 2nd paragraph). For the tables in the main text, we only presented statistics with a p-value <0.05. We have now inserted the descriptive statistics and p-values from the statistical online resources to the main article file (see Table 2 in Results section, pag.8). I recommend expanding empirical research and including survey results in this research, perhaps using some statistical software. This extension of empirical research can better support the discussions and conclusions presented. OUR RESPONSE: We have now strengthened the statistical analysis by including p-values, confidence intervals for the association of each variable, and weight of non-respondents (see Tables  2 and 3 in main text). We have also strengthened the discussion of the internal and external validity of our results (see the bullet points in pag.2, and Discussion, final three paragraphs in pag.12). I appreciate the paper as well structured both from the point of view of the format and from the scientific point of view, I strongly recommend the detailed presentation of the empirical research and the implementation of the suggested changes in order to continue the path to publication. OUR RESPONSE: We have now presented more details of the empirical research carried out previously on this very sample of physicians (see Introduction, last but one paragraph in pag.4). We have also strengthened the statistical analysis for this paper (see the Variables and Data analysis section final two paragraphs (pag 7 after  Tables 2 and 3). We have also added a description at the beginning of the Results, a geographical map depicting our sample in the two states in Online Resources 2 (see Fig.S1), and added a table in the statistical annex summarising the sample's characteristics (Table S1 in Online resources 2).

VERSION 1 -AUTHOR RESPONSE
Lastly, I believe the analysis could enrich by including a test of means difference between territories. OUR RESPONSE: As our analysis for this paper only considers qualitative variables (and therefore means difference cannot be tested), following Revi.2's suggestion we have now applied the statistics from the chi-square test as a complement to the confidence interval analyses (see Table 2 and 3). Reviewer: 3 Comments to the Author: This study provides some intriguing survey data on perceptions of Brazilian doctors regarding demand for their services during the COVID-19 pandemic. The study is well written and uses simple but rudimentary descriptive methods to present its findings. It uses proxies for capturing perception of doctors, which limits its utility in two ways: perceptions can depart from reality and the specific proxies chosen here may contain some measurement error in and of themselves, especially the supply proxy items. That said the study is a good addition to existing body of literature on pandemic impacts. However, I would caution authors against reaching more sweeping interpretation than their data and methods allow. I am not sure the study results permit a clear inference about differential impacts of the pandemic on labor markets in private vs. the public sector, absent more information on the impacts on diverse settings within the public sector (e.g., primary care, urgent care elective care and inpatient settings). I discuss these (and other more minor) issues in more detail below: 1. It is not clear that the text of the question used to proxy physician supply is cleanly worded ("availability of physicians for recruiting"; Table 1 row 5) or directed at the right kind of subjects to elicit accurate information. Physicians are likely to have a good idea of demand for their own work but supply and recruitment difficulties would be better known by hiring administrators and executives responsible for running hospitals and clinics. Authors briefly allude to this issue in limitation but it needs to be emphasized more directly in interpreting their results. The anomalous finding of price stability given the supply-demand mismatch may be explained by an erroneous measurement of supply.
Our response: This is a valid point, and we accept the translation of this question from Portuguese into English was not accurate. We have now changed it to 'Availability of new doctors to fill positions' (see table 1). We also acknowledge in the Discussion that such information on supply of doctors may not have been known to physicians (see Limitations, pag.12, 2nd paragraph), and included the answer 'Unknow' as answers in the tables. As for the anomalous finding on price stability, we offer a range of possible explanations, and have now added 'erroneous measurement of supply' to that list (same paragraph in Limitations at the end of Discussion). 2. The text of the survey question for A&E remuneration differs between Table 1 (row 5, column 2) and the Datafolha instrument provided in the supplementary appendix (Q 34). The latter question does not refer to A&E shift but to specialty wages. Please address the discrepancy. OUR RESPONSE: This has now been addressed. The discrepancy was due to poor translation of the Datafolha questionnaire into English (see Q34 in Table 1 and Q34 in Online resources 1). 3. Please label supplementary tables using a different convention e.g., S2 or A2 to distinguish from tables in the main manuscript. OUR RESPONSE: This has now been addressed, and the supplementary tables from the online resources 2 have now been called S1, S2 and S3. 4. The denotation for confidence interval is "CI" in the text and "IC" in the tables. Please make it consistent. OUR RESPONSE: This has now been addressed in the new Table 2 and 3. 5. Were any formal statistical comparisons (t-test between group means) between responses for the two states made? I see it alluded in methods section but don't see it discussed in results or described in tables. Our response: As in this manuscript we are only considering qualitative variables, t-test for means values cannot be calculated; however, we have now applied the statistics from the chi-square test as a complement to the confidence interval analyses (see the new Tables 2 and 3). 6. I understand that responses to survey items included options to choose "unknown" when the respondent was unsure as to the main response options. However, I do not see this option listed in the descriptive data tables. Was this because no respondent picked that option? That would be unusual, especially for supply proxies, where the text is somewhat ambiguous. Relatedly, some of the cells in Table 3 do not add to 100%. For example, response options for overall sample regarding the perception of availability of doctors do not add up to 100% (increased 44.8, decreased 27.7, no change 19.2). Also, total number of respondents is listed as 1181 in some tables, whereas the Sao Paulo (632) and Maranhao respondents (551) total up to 1183. Please clarify these anomalies. Our response: This is a very important point; in our previous analysis we had chosen (erroneously) to disregard the "unknown" category. We have now corrected this in the revised analysis, and inserted this category in the tables. As a result, also the key statistical results have been changed accordingly through the manuscript. We thank the reviewer for spotting this. Table 3 has also been corrected, as well as overall number of physicians included. Apologies for this oversight. 7. From the Datafolha instrument, it is evident that respondents were also asked to quantify the intensity of their initial responses to demand and supply proxy items on a 10-point scale (Q31a, 31b, 33a, 33b). Why was this information not utilized to further evaluate and contextualize their initial responses? OUR RESPONSE: Response rate to those questions were too low to allow meaningful significant statistical analysis. We noticed physicians struggled to attribute a 0-10 score to the intensity of changes observed during the pandemic. 8. Authors interpret their findings as providing evidence that pandemic had differential effects on public sector and private sectors labor markets. I am not so sure. It may also be that pandemic had major impacts on specific acute care clinical settings like hospitals (where demand accelerated due to exploding COVID-19 related morbidity/mortality) regardless of whether these were public hospitals under SUS or private hospitals. Moreover, as authors themselves state, many SUS hospitals are run by private non-profit entities that hire contractual workers. Did SUS facilities that primarily provide elective or primary care buck the trend of reduced work opportunities? If so, that would support the theory of differential impacts but I am not sure whether SUS-related response data for demand proxy items was disaggregated by facility setting type OUR RESPONSE: We accept this point about some public hospitals being run private entities (known as Social Health Organizations -OSS). However, as a substantial proportion of doctors concomitantly work on different hospitals and services, and the public/private nature of the entity running the services is not always know to them, it was not possible to collect such information. We have now acknowledged such limitation in the Discussion (Limitations, pag.12, 1st paragraph). 9. Some statements in the discussion are ambiguously worded and inconsistent with results: Demand and supply of doctors in Sao Paolo and Maranhao appear to have experienced opposite pandemic effects…., Page 10, Lines 59-60). Please review and reword. OUR RESPONSE: This sentence has now been reworded (see the Discussion, pag.11, 2nd paragraph). We would like to thank the three reviewers for spotting the inaccuracies in our earlier draft, and for offering solutions to fix them.

GENERAL COMMENTS
My comments have been amended. I recommend the paper for publication.

REVIEWER
Bhandari, Neeraj University of Nevada Las Vegas, Healthcare Administration and Policy